Method for encoding image pixels a method for processing images and a method for processing images aimed at qualitative recognition of the object reproduced by one or more image pixels

ABSTRACT

A method for encoding pixels of digital or digitized images, i.e., images consisting of a set of image dots, named pixels in two-dimensional images and voxels in three-dimensional images, each of said pixels or voxels being represented by a set of values which correspond to a visual aspect of the pixel on a display screen or in a printed image. According to the invention, the pixels or voxels of at least one portion of interest of the digital or digitized image or each pixel or voxel of the set of pixels or voxels which form the image is uniquely identified with a vector whose components are given by the date of the pixels or voxels to be encoded and by the data of at least one or at least some or of all of the pixels around the pixels to be encoded and arranged within a predetermined subset of pixels or voxels included in the whole set of pixels or voxels which form the image.

A method for encoding image pixels, a method for processing images and amethod for processing images aimed at qualitative recognition of theobject reproduced by one or more image pixels

The invention first relates to a method for encoding pixels of digitalor digitized images, aimed at making the information content of eachpixel available to automatic image processing systems, particularlydesigned for image recognition with reference to the objects reproducedtherein.

The term digital is intended to define an image obtained by imagingapparatuses whose output is an image in digital format, i.e. digitalcameras, Nuclear Magnetic Resonance imaging apparatuses, ultrasoundimaging apparatuses and other imaging apparatuses.

The term digitized images is related to images obtained by substantiallyanalog systems, providing an analog image which is further scanned bymeans of devices known as scanners, regardless of whether the latter arehardware devices, i.e. devices for reading an image which is physicallyprinted on a medium, or software devices, i.e. designed to sample animage, provided in the form of set of signals and to turn it intodigital signals.

The increasingly extensive use of digital images allows to have saidimages in the form of numerical data, ready for further processing.

Regardless of whether the image is directly obtained digitally oracquired in an analog manner, by traditional systems, to be furtherdigitized, e.g. scanned by digitizing devices, i.e. scanners, a digitalimage is composed of a set of image dots, the so-called pixels, whichmay have different brightness conditions, i.e. different gray scaletones and, in color images, different colors. Each pixel of an imagealso has a well-defined position whereby the digital image may berepresented by a two- or three-dimensional matrix of elements Pi,j, eachcorresponding to a predetermined pixel of the pixel set that forms theimage, the element Pi,j being a variable which assumes the brightnessand/or color value associated to the specific pixel.

When the range is restricted to black and white images, the differentpixel-associated brightness values are represented by a gray scaleextending from black to white through several different intermediategray levels, whose number may be user-defined, based on the capabilitiesof the digitized imaging apparatus and/or of the display device.

In the case of a three-dimensional matrix, the discrete image unitelement is generally referred to as a voxel and the three-dimensionalmatrix is composed of elements Vi,j,k.

Therefore, from the technical point of view, a digital image has aunique equivalent in the form of a data matrix which forms a virtualimage and, as such, has a structure that is potentially adapted forimage processing by systems or methods which use algorithms, whetherprovided by software loaded in computers or by dedicated hardware foraccomplishing specific functions on the image matrix.

Nevertheless, from a logical point of view, the information contained ineach isolated pixel Pi,j or voxel Vi,j,k provides nothing but the simpleindication of its brightness value, i.e. the gray scale valuecorresponding thereto, hence it has no meaning wherefrom imageinformation may be extracted, and only acts as data for controlling thedisplay device, which may be and actually is handled during the imagingprocess to adjust the general aspect of the image, e.g. contrast and/orbrightness and/or specific color as defined based on user-selectedfunctions, depending either on objective or subjective data.

The image obtained thereby, as regards the shapes and characteristics ofthe objects reproduced by pixels in the different image portions derivesfrom the relation of each image pixel with the surrounding pixels.Therefore, in order to allow image processing to not only adjust theindividual pixels to improve the quality of the displayed image, it isnecessary to define the relations between each pixel and the pixelsaround it. At present, no rule exists to determine such relations,except those defined on the basis of assumptions or presumptivelydefined rules, based on the specific characteristics of the objectsreproduced by the image.

Therefore, the invention is based on the problem of providing a methodfor encoding image pixels, which allows to account for the relations ofeach pixel with the pixels around it, substantially regardless of thepeculiar characteristics of the object specifically reproduced in theimage, i.e. a method that can be used to provide an image data set,particularly adapted for an image processing procedure aimed atrecognizing at least some characteristics of the objects represented inthe image, as well as of the shapes of these objects.

An additional object is to provide an encoding process as mentionedabove, which is simple and requires neither complex processing steps,nor long processing times, and does not cause the hardware required tostore the encoded data to be overloaded.

The invention achieves the above purposes by providing a method forencoding pixels of digital or digitized images, wherein each pixel ofthe pixel set which forms the image is uniquely identified with a vectorwhose components are given by the data of the pixel to be encoded and bythe data of at least one or at least some or at least all of the pixelsaround the pixel to be encoded, which pixels are disposed within apredetermined subset of pixels included in the total set of pixels thatform the image.

As a first approximation, the components of the pixel identifying vectorare determined by selecting, as pixels surrounding the pixel to beidentified, all the pixels that are directly adjacent to said pixel tobe encoded.

The components of a pixel identifying vector may be also extended to atleast one or at least some or all of the pixels which surround thepixels directly adjacent to the pixel to be encoded.

Theoretically, no limit is imposed to the number and position of pixelsaround the pixel to be identified, which may be used to define thecomponents of the pixel identifying vector.

The components of the identification vector corresponding to the pixelto be identified and to the surrounding pixels are arranged in such amanner as to reflect the arrangement of the pixels within the pixelmatrix which forms the image, with reference to a predetermined pixelreading sequence, for forming said vector.

Particularly, the components of the identification vector, correspondingto the pixel to be identified and to the surrounding pixels are arrangedin such a manner as to correspond to the distance relation of saidpixels with one another and with the pixel to be encoded, with referenceto a predetermined pixel reading sequence, for forming said vector.

The components of the identification vector are arranged in such amanner that the pixel to be identified has a central position whichcorresponds to the one taken in the image pixel set, obviously asrelated to the surrounding pixels, which pixel set has been selected fordetermining the identification vector components.

The methods includes the generation of an identification vector for eachpixel which forms the digital or digitized image.

By this arrangement, the virtual image composed of a data matrix whichcorresponds to a set of virtual pixels, i.e. a set of data having thesame position as that of real, actually displayed image pixels, isturned into a matrix in which each element has, at a certain pixellocation, the identification vector therefor, which in turn has thenumerical structure as defined above.

Since the identification vector includes components given by the dataassociated to a certain predetermined number of pixels surrounding thepixel to be identified, the latter is defined not only by the numericalvalue corresponding to the brightness thereof, but also by the numericalvalues that correspond to the brightness of the surrounding pixels,which were selected to form the identifying vector components. Hence,the matrix of pixels, i.e. of brightness data associated to the pixelsis changed into a set of vectors.

The pixel identifying vector may be also extended to other components,e.g. the values of the selected pixels and of the pixel to be identifiedat different instants of time. This is advantageous when, for instance,different digital or digitized images of the same frame, acquired atdifferent instants, are available. In this case, the vector will beassociated to a succession of different sets of components, comprisingthe value of the pixel to be identified and the values of the selectedpixels around it, each set being determined by an image acquired orreferred to the same frame at different acquisition instants.

Preferably, the component sets are ordered within the identificationvector in a succession corresponding to the time sequence of capturethereof.

The above clearly shows that the encoding method of the invention,besides allowing to identify each pixel based on its numerical value andon the relation of said pixel to a certain number of surrounding pixels,also extends this identification to the time variation of said pixel tobe identified and to the time variations of the relations of said pixelto be identified to the selected surrounding pixels. Thanks to theencoding method according to the invention, a numerical description maybe provided for each image pixel, even for sequences of imagesrepresenting moving objects, any change caused by the movement of theobject being contained in the identification vector.

The method as described above may be easily implemented both for two-and three-dimensional imaging. In the latter case, the number ofcomponents of the identification vector considerably increases, in acubic progression, if all the pixels which form the increasingly distantshells around the pixel to be identified are to be accounted for.

The pixel selection pattern around the pixel to be identified and whosedata shall form the components of the identifying vector may varydepending on actual needs.

According to a further embodiment of the method for encoding imagepixels, instead of considering as the relevant minimum image surface tobe coded only one pixel a group of adiacent pixels may be considered. Ina square sub matrix of pixels for coding, the minimum image area to becoded can be formed by the four center pixels of the said sub matrix. Inthis case the at least one value assigned to the group of four pixels inthe coding vector or matrix is formed by a linear or non linearcombination of the brightness of the said four adiacent pixels.

The value of the group of pixels to be coded as the minimum image areamay be a combined value comprising several parameters each oneconsidering for example a linear or non linear combination of thebrightness of the said four grouped pixels, such like a mean value ofthe brightness of the four grouped pixels, and statistical values suchas brightness variance of the four grouped pixels. As numerical valuesof the said grouped pixels considered a minimum image area also thecolour values of the four grouped pixels might be considered if theimage is a clour image and even the variation relatively to time of thesaid brightness and cloir values. Also color values of the groupedpixels might be in the form as a linear or non linear combination of thecolor values of the single pixels of the group and/or to valuescorresponding to statistical parameters such as variance or the like.

The variation of the brightness values and or of the color values of thegrouped pixels might be considered as the variations relatively to timeof the linear or non linear combinations, such as mean values or otherstatistical parameter elatively to time.

In the identification or coding vector for such a minimum image areaformed by a certain number, for example four grouped pixels, mightcomprise similarly to the above described identification vector thebrightness and or color values and/or their variation relating to timeof a certain number of surrounding pixels.

In this case of a minimum image area to be coded to alternatives mightbe choosen. In the first one each selected pixel surrounding the groupedpixels forming the minimum image area to be coded is considered alone orper se.

In a second alternative embodiment the selected surrounding pixels aregrouped similarly to the ones to be coded thus forming surroundingminimum image zones the values of this surrounding image zones beingdefined by linear or non linear combination or by other statisticalfunctions of the brightness of the signle pixels being grouped as aminimum image surrounding zone.

The above disclosed embodiment has the advantage to reduce calculationtime in the case the pixels are very small relatively to the subjectreproduced by the image, so that within the area of only one pixel novariation of the structure of the imaged subject may occur.

The invention also relates to an image processing method, particularlyaimed at recognizing objects and/or object shapes, in an image in whichpixels are encoded into identification vectors.

The method for processing digital or digitized images includes thefollowing steps:

Encoding different digital or digitized images into pixel identifyingvectors, with image objects being identified for the selected types andqualities, and associating each identification vector to thecorresponding known type of object or to the known quality that areactually reproduced by each pixel identified by said identificationvector, with reference to a predetermined list of different types orqualities.

Generating a teaching database for an expert processing system,comprising the binomials Formed by said identification vectors and bythe associated type or quality of the reproduced object;

Actually teaching the expert processing system, by entering the teachingdatabase therein;

Encoding the pixels of a not otherwise evaluated image by identificationvectors associated to each pixel;

Entering said image pixel identifying vectors in the processing systemto obtain, at the output of said processing system and as a result ofthe processing, the type or quality of the object represented by theimage pixels.

The process of recognizing a quality or a type of object and assigningit to a pixel identifying vector is not complex, because it only is acomparison. The difficulty lies in the great number of pixels, hence ofthe vectors to be compared, and in that the vectors have a considerablenumber of components. When accounting for the processing time, howeverfast, of each step in which the identification vector for a pixel to berecognized is compared with the identification vectors for pixelsincluded in the basic knowledge or teaching database for the expertsystem, total times are extremely long and unacceptable in practice.This field of application is particularly suitable for particularprocessing algorithms and more precisely for the well-known neuralnetworks.

Nevertheless, it shall be understood that, while the use of a neuralnetwork as a processing system is the preferred embodiment, theinvention is not limited to said algorithm, but may address any type ofalgorithm for comparing the identification vectors of the image pixelsto be processed with the teaching database, such as a discretizingalgorithm which makes a dull comparison and decides whether theidentification vector belongs to one or the other type of object orfeature amongst the various possibilities.

As will be apparent hereafter, the pixel encoding method according tothis invention provides highly reliable and accurate results, i.e.higher than is currently expected.

These algorithms can rapidly converge toward the correct solution or toa solution to be statistically considered as the best possible solution,by also accounting for the considerable variances of the identificationvectors for the same object type or quality.

The result provided by the expert processing system may be viewed bysimply printing or displaying a list. As an alternative thereto or incombination therewith, the processing result may be highlighted byassociating a color to each type or quality and by representing thesolution over the digital or digitized image with each pixel of thedigitized image being assigned the color of the corresponding type orquality of the represented object, as determined by the expertprocessing system.

It shall be noted that the teaching step, based on either differentimages of the same frame at different times or on images of differentframes or objects whose type or quality is one of the predeterminedoptions, allows the expert processing system, particularly a so-calledneural network to learn what the aspect of the identification vectorshould be for a particular object or a particular quality with thehighest variance of this aspect.

It shall be also understood that the recognition of the reproducedobjects and/or qualities is independent from the global processing ofthe image and that it is performed pixel after pixel with no referenceto what the pixel set represents within the image.

These two characteristics are of basic importance. By providing as muchvariance as possible for the identification vectors for pixels thatrepresent a certain object type or quality, the processing system isallowed to recognize more accurately and reliably whether anidentification vector, hence a pixel, belongs to a certain type ofobject or to a certain quality.

Pixel-based processing allows to substantially unlink the recognition ofa pixel identifying vector as belonging to a certain object type orquality from the imaged subject.

The image processing method provides other advantages.

A first additional advantage consists in that the list of object typesor qualities may be modified anytime, i.e. restricted or extended,without affecting the previous teaching process, by simple integrationin the teaching database for the processing system. It is also possibleto restrict image processing to only recognize some of the types orqualities of the imaged object among all the qualities or types of theteaching database, without affecting any further extension thereof.

Moreover, with use, the database including the knowledge acquired by thesystem may be increased, thereby improving knowledge, expertise hencereliability and accuracy of the processing system.

Objective evaluation of pixel characteristics allows to avoid any humanimage recognition and interpretation errors.

By a targeted selection of the object types and/or qualities to berecognized and the generation of adequate teaching databases, the sameprocessing system may be used to accomplish different functions.

Besides simply recognizing object types or qualities, it is possible togenerate teaching databases which allow to correct defocused images oraccurately overlay two digital images of the same subject, acquired withdifferent imaging methods, e.g. by Nuclear Magnetic Resonance imaging,ultrasound imaging or radiographic or photographic imaging.

Yet another advantage provided by the recognition method of thisinvention consists in allowing to limit image definition duringacquisition, thereby obtaining identical or even better results asregards the possibility to evaluate the acquired image thanks to abetter and more accurate recognition allowed by the method, as comparedwith human eye potential. This provides an important advantage, a lowerresolution involving a reduced duration of imaging, e.g. by NuclearMagnetic Resonance or by ultrasounds or other similar means. This notonly allows to reduce the costs required for fast imaging and imagereconstruction apparatuses, but also has positive implications, namelyfor the comfort of the patient, who does not have to keep still for verylong times.

A particular application of the image recognition method of theinvention consists in the automatic recognition of tissue types from thediagnostic images acquired by Nuclear Magnetic Resonance imaging,ultrasound imaging, radiography, etc.

In this case, the method includes the following steps:

-   -   generating a database for teaching the expert processing system        based on pixel identifying vectors obtained by encoding        well-known digital or digitized diagnostic images, whose pixels        have been previously associated by an expert to a tissue type,        which pixel identifying vectors are uniquely associated to the        corresponding known tissue type;    -   Teaching the expert system by entering the teaching database        therein, as well as the modes associated to the processing        algorithm type;    -   Encoding an image or a time sequence of images of the same        frame, into pixel identifying vectors according to the method of        this invention, which image or sequence is digitally acquired or        has been digitized but not interpreted;    -   Entering pixel encoding vectors for the non interpreted image        entered into the processing system, which processes the tissue        type or the tissue quality for each pixel identifying vector and        displays the result thereof.

Particularly, the result is displayed as color assignments, wherepredetermined colors are assigned to the different types of object or tothe different qualities of the pixels which have been found to belong toan object type and/or quality.

It shall be noted that in this case the method has no diagnosticfunction, but generates considerably reliable indications for thephysician or for the technical personnel responsible for the evaluationof the acquired image. No direct treatment suggestion is provided, butsimply an indication of a type of tissue which is highly likely to befound in the image. The actual and total certainty of the result forfinal diagnosis requires both the image to be read and interpreted byqualified personnel and other cross-checks to be performed by otherdiagnostic methods.

Nevertheless, as will be more apparent from the following, it is noteasy to read and interpret a diagnostic image, like a radiographicplate, an ultrasound image or a Nuclear Magnetic Resonance image,particularly when the diseases reproduced in the image have a very smallextension. The instrument provided by this invention allows to reliablysignal potential pathologic elements, while reducing the risk ofmisinterpretation, or preventing the same elements from beingmisinterpreted or even from not being seen by the physician or qualifiedpersonnel.

Further improvements of the invention will form the subject of thesubclaims.

The characteristics of the invention and the advantages derivedtherefrom will appear more clearly from the following description of anon limiting embodiment, illustrated in the annexed drawings, in which:

FIG. 1 shows a simplified digital or digitized image pixel encodingscheme according to this invention, in a two-dimensional space.

FIG. 2 is a scheme as shown in FIG. 1, referred to a three-dimensionalimaging space.

FIG. 3 shows a scheme of the identification vector for the same pixel ina time sequence of digital or digitized images.

FIG. 4 shows a block diagram of the inventive image processing method,which operates based on the pixel encoding method as shown in theprevious figures.

FIG. 5 shows a data table and a chart of the image processing resultsfor the image as shown in FIG. 8 or in FIG. 11, obtained by NuclearMagnetic Resonance imaging, and in which the processing system wasoperated with a teaching base only comprising two types of tissues, i.e.benign tumor tissues and malignant tumor tissues.

FIG. 6 shows a data table, and a corresponding chart of the imageprocessing results for a Nuclear Magnetic Resonance image, as seen inFIG. 8 or in FIG. 11, in which the processing system used a teachingbase comprising three types of tissues, i.e. benign tumor tissue,malignant tumor tissue and normal tissue.

FIG. 7 shows a data table and a corresponding chart of the imageprocessing results for a Nuclear Magnetic Resonance image, as seen inFIG. 8 or in FIG. 9, in which the processing system operated with ateaching base comprising four types of tissues and an additional objecttype, i.e. benign tumor tissue, malignant tumor tissue, normal tissue,muscular tissue and background image.

FIG. 8 shows a first Nuclear Magnetic Resonance image, in which thepresence of a benign tumor tissue is outlined by a white ring.

FIG. 9 shows the same image as FIG. 8, but processed with the method ofthe invention and with the mode in which only benign tumor tissues arerecognized, in which the system has displayed the result on theoriginally acquired image by assigning a different aspect to the pixelswhich have been found to belong to the one or the other tumor tissue.

FIG. 10 shows the same image as FIGS. 8 and 9 once it has been processedaccording to this invention, the system having been taught to recognizethe four types of tissues and the background, and the system having beenprompted to display the originally acquired image by assigning differentaspects to the pixels which have been found to belong to the differenttissue types and/or to the background.

FIGS. 11 and 12 are another Nuclear Magnetic Resonance image containinga malignant tumor tissue, as outlined by the white ring, and an enlargedimage of said region containing the tumor tissue respectively.

FIG. 13 is the same image as FIG. 11, representing the result of imageprocessing according to this invention, operated for recognition anddifferentiated display, on the same acquired image, of five differentobject types, including four tissue types (benign tumor, malignanttumor, normal and muscular) and a background type.

FIG. 14 shows the inventive processing method in combination with stepsto display pixels corresponding to a resolution below the human eye andto a resolution level with or worse than that of the human eye.

FIG. 1 shows the inventive encoding method in a highly simplifiedmanner, in the case of a two-dimensional digital or digitized image,i.e. consisting of a set of pixels (image unit elements).

The example shows a pixel of the pixel set, denoted as 5, which isdesigned to be encoded in such a manner as to make the informationavailable for any type of treatment, particularly for processing.

In this case, the image may comprise any number of pixels and the stepsof the method, as shown with reference to the pixel 5 of FIG. 1 areexecuted for every pixel of the image. In the inventive encoding method,the pixels around the pixel are used to form an identification vectorfor the pixel 5. The surrounding pixels may be selected according topredetermined rules which may lead to different selections ofsurrounding pixels, as components of the identification vector, both asregards the number of the surrounding pixels to be selected ascomponents of the identification vector and as regards the location ofthese surrounding pixels relative to the pixel to be encoded, in thiscase to the pixel 5.

One of the most obvious selections consists in using, as components ofthe identification vector for the pixel to be encoded, all the pixelsdirectly adjacent to the pixel to be encoded, that is, in the notationreferred to pixel 5 of FIG. 1, the surrounding pixels 1, 2, 3, 4 and 6,7, 8, 9.

In black and white or gray scale images, the value represented by eachpixel is given by a brightness value of the corresponding pixel, i.e. agray value in a gray scale extending from white to black through acertain number of intermediate levels, which may have a different numberof gray tones, depending on the quality of the digital image withrespect to the color resolution of the imaging apparatuses.

Depending on the type of color encoding, in color images, each pixel mayalso have one variable for indicating the color to be assigned thereto.

The example of the following figures will be limited to black and whiteor gray scale images, to better explain the steps of the method. Theextension to pixel color indicating variables is an obvious step forthose skilled in the art, which eventually involves the presence of agreater number of components in the identification vector.

FIG. 1 shows the structure of the identification vector for a pixel withreference to pixel 5.

The vector comprises, in the same pixel indexing sequence within thepixel matrix, all the pixels which constitute the vector components,starting from pixel 1 and ending with pixel 9. In this case, the pixel 5located at the center of the pixel matrix appears to occupy a centralplace in the sequence of the identification vector components.

The manner in which the pixels designed to form the identificationvector for a pixel to be encoded are indexed is not relevant per se;what is important is that the selection of this manner is consistentlyand accurately followed for all encoding processes, otherwise nocomparison between two pixel vectors would be possible, since the vectorcomponents would have different arrangement orders.

Therefore, it shall be noted that, for this encoding operation, theidentification vector for pixel 5 does not only contain gray-scalebrightness, i.e. pixel aspect information about the pixel to beidentified, but also brightness information about the pixels around it.

This vector structure is based on the acknowledgement that the contentof an image is not recognizable based on the aspect of an individualpixel, but based on the relation between the aspect thereof and theaspect of the surrounding pixels. In practice, each image dot is notimportant per se, unless it is evaluated with reference to the aspect ofthe surrounding dots or areas. Even from the visual point of view, whatis shown in an image is recognized on the basis of a relative evaluationbetween the different areas of the image.

As mentioned above, the selection of the surrounding pixels to createthe identification vector for the pixel to be encoded is not governed byany specific rule.

For example, it is possible to increase the number of surrounding pixelsto be accounted for to generate the identification vectors, by using, asvector components, at least some or all of the pixels of pixel ringssurrounding the central pixel to be encoded, at increasing distancesfrom the central pixel to be encoded.

As in the illustrated example, it is possible to account for all or someof the pixels of the pixel ring which externally surrounds the pixelring directly adjacent to the central pixel 5 i.e. the pixels 1, 2, 3,4, 6, 7, 8, 9.

In this case, the number of vector components increases drastically, andoverloads the identification vector processing conditions. If theidentification vector for pixel 5 is arranged to comprise, for instance,all the pixels that externally surround the illustrated 3×3 pixelmatrix, the number of identification vector components increases from 9to 25 components.

In this case, processing might obviously provide a more precisesolution.

Furthermore, said additional pixels at a longer distance from the pixelto be encoded may be suitably weighted, possibly also in a differentmanner from each other, to attenuate the effect thereof on theidentification vector.

FIG. 2 shows the situation of a three-dimensional image, in which thecentral pixel 14 is encoded by an identification vector which has, ascomponents, the values of all the pixels directly surrounding it andsubtending a 3×3×3 cube, whereby it includes 27 components.

The considerations proposed for the two-dimensional embodiment alsoapply to the three-dimensional embodiment. This shows more clearly theoverloaded processing of the identification vector when other pixels areaccounted for, e.g. those of a three-dimensional cubic pixel shell whichencloses the illustrated cubic set. In this case, the number ofcomponents increases in a cubic progression, from 27 pixels to 125pixels, if all the pixels of a 5×5×5 cube are considered in theidentification vector.

According to an additional characteristic of the invention, the pixelvector encoding method allows to also integrate the behavior throughtime of the pixel under examination in the identification vector, when asequence of images of the same frame are available.

The sequence of images may be composed, for instance of frames of amotion picture or of individual images of the same frame as taken atsuccessive instants. An example of imaging of the same frame atsuccessive instants consists in diagnostic ultrasound imaging ofcontrast agent perfusion. In this case, the perfusion of contrast agentspushed by the flows of vascular circulation is imaged by injectingcontrast agents in the anatomic part under examination at the instantTc, and by subsequently imaging the same part at predetermined timeintervals. Time variations of the image allow to check the presence ofcontrast agents, after a certain period from the injection instant.These images may provide useful deductions and/or information to checkthe presence of vascular and/or tumor diseases.

In the above case, the recognition of the reproduced object is not onlybased on the aspect thereof, but also on the time variation of saidaspect. Therefore, the pixel vector encoding process aimed at including,in the identification vector for each pixel, all the data characterizingthe quality or type of the object reproduced by an image pixel mustespecially account for the time variation of the encoded pixel.

In this case, the identification vector for a predetermined pixel, e.g.pixel 5, with reference to the embodiment of FIG. 1, contains a set of 9components, relating to pixels 1 to 9 in the proper time sequence, foreach instant whereat the corresponding image has been captured.

The illustrated embodiment show six images of the same frame as acquiredat the instants T=0, T=1, T=2, T=3, T=4, T=5. Moreover, the embodimenthas been developed with reference to the acquisition of a sequence ofultrasound images of the same anatomic part, performed after theinjection of contrast agents. The instant TC whereat contrast agents areinjected is denoted by the arrow TC.

The identification vector accounts for pixels 1 to 9 of the three imagesacquired at the instants T=0, T=1, T=2 and in these conditions theidentification vector already has 27 components. When all the 6 imagesof the time sequence are accounted for, the components will increase to54. This only relates to an encoding process which accounts for a 9pixel matrix, in which the pixel to be encoded is the central pixel.When the encoding process is intended to include the pixels surroundingthe pixel to be encoded in a 5×5 pixel matrix, i.e. having 25 componentsof the identification vector for each instant, provided that all theimages of the sequence are considered, each identification vector willhave 150 components.

In the case of a three-dimensional image as shown in the example of FIG.2, in which the identification vector for a pixel, with surroundingpixels of a cubic 3×3×3 pixel space, has 27 components, the encodingvector according to the example of FIG. 3 would include, for all siximages of the image time sequence, 162 components. If encoding isextended to a 5×5×5 pixel three-dimensional space, the components of theidentification vector for the pixel will increase to 750.

It shall be noted that while the inventive encoding method may befollowed in a simple and fast manner, it allows to identify thecharacteristics of an image pixel through its value as well as inrelation with its surrounding pixels, and also with reference to timevariations of the pixel to be encoded and of the surrounding pixels.

It shall be further noted that this encoding method only accounts forpixels, and of a substantially restricted examination field, which isindependent from the subject of the image and of the encoding purpose.

Obviously, encoding times for an image or a sequence of images strictlydepend on the image size, in terms of number of pixels.

According to a further embodiment of the present pixel coding method,instead of considering an image formed by single pixels, for codingpurposes a minimu image area of different size may be defined which isformed by a more than one adiacent pixel.

So referring to the above described method and particularly to FIG. 1 to3, with 5 in FIG. 1 and 14 in FIG. 2 there is not identified a singlepixel but the said minimum image area formed by a certain number ofpixels, for example by a matrix of four adiacent piels or of 9 adiacentpixels.

In this case the value of the said minimum image area formed by groupedadiacent pixels may be calculated as a linear or non linear combinationand as the value of one or more statistical functions of the of thebrightness and or color values of the single pixels grouped together forforming the said minimum image area.

A simple example would consider the mean of the brightness values of thesingle pixels forming the minimum image area the variance thereof. Tothis the information relating to the color of the pixels could be addalso in the form of a mean and or of a variance value. Furthermore alsoin this case the time dependency of the values relating to brightnessand/or color could be add similarly as disclosed above relating to FIG.3. In this case the time dependency of the combination and of thestatisitical function of the brightness and or color values of thesingle pixels will be considered, i.e. the variation relatively to timeof the mean value of the brightness or color values of the single pixelsforming the minimu image area and or the variation relatively to time ofthe value of the statistical fuction.

As selected neighborhood pixels for the pixel vector identifying thesaid minimum image area, there might be the choice between toalternatives.

In one alternative the selected piels of the neighborhood are consideredper se.

In the second alternative also the neighborhood pixels are groupedtogether for forming selected minimum image neighborhood areas. Toexplain more clealrly the idea, if FIG. 1 or 2 are considered the imagearea indicated as pixel 1 to 4 and pixel 6 to 9 in the example of FIG. 1are not single pixels as described in the previous example but areformed by a certain number of adiacent pixels. This certain number ofpixels might be the saome as for defining the minimu image area to becoded or it might be a different number of pixels as the one grouped forforming the minimum image area to be coded.

The advantage of such a oding method condidering minimu image areacomprising more than ome pixel will speed up the coding computationaltime and is useful and will give not rise to errors particulallry in thecase the subject being imaged does not show any structural change withinan area of several pixels, or the image resolution is o low that nostructural variation can be sensed and shown within the area for two ormore adiacent pixels forming the image.

In the strictly diagnostic field, for instance, in which, due topractical and economic reasons, the number of image pixels is generallylimited to 256×256 pixels for a two-dimensional image, encoding isextremely fast. In this cases a coding according to the method asdisclosed with reference to FIGS. 1 to 3 may be used without sufferingby a too high duration of the coding procedure.

The encoding method as disclosed above, which includes the informationabout the aspect of a pixel with reference to the surrounding pixelsand/or the time variations thereof, allows image processing methodswhich substantially require a recognition of the object type or of thequality of what is reproduced by the pixel, and allow to automaticallyrecognize the type of object and/or the quality thereof by the sameprocessing system or software.

FIG. 4 is a block diagram of a method of processing digital or digitizedimages, operating on the basis of the previously described encodingmethod.

The processing method includes two steps: teaching the processing systemand processing.

Processing is performed, per se, by an algorithm which is basically analgorithm for executing comparisons between a database that includes acertain number of identification vectors for pixels associated to thetype of object or to the quality corresponding thereto and theidentification vectors for pixels of an image to be processed.

As a result of the comparison between the identification vector for eachpixel of the image to be processed and the identification vectors forthe pixels included in the database, the comparison algorithm assigns toeach pixel encoding vector within the image to be processed, or to apredetermined portion thereof, the most appropriate or probable or theclosest type of object or quality of the pixel identifying vectorincluded in the database.

The processing algorithm may be a simple discriminating algorithm, forinstance an LDA algorithm (Linear Discriminant) (S. R. Searle, 1987,Linear Models for unbalanced data, New York, John Wiley & Sons) or amore complex algorithm, e.g. neural networks. The image processingprocedure to be used is a typical application for neural networks, i.e.an application in which a very great number of typically simpleoperations is required, and which finds no exact numerical solution dueto the considerable number of identical processing steps to beperformed. In practice, a dull execution of the steps for comparingidentification vectors for the image pixels to be processed, with theidentification vectors for the pixels of the reference database, wouldrequire so lung computing times as to be unacceptable.

A number of neural networks might be used, for instance those known as:MetaGen1, MetaGen, MetanetAf, MetaBayes, MetanetBp, MetanetCm (M.Buscema (ed), 1998, SUM Special Issue on ANNs and Complex SocialSystems, Volume 2, New York, Dekker, pp 439-461 and M. Buscema andSemeion Group, 1999, Artificial Neural Networks and Complex SocialSystems [in italian], Volume 1, Rome, Franco Angeli, pp 394-413 and M.Buscema, 2001, Shell to program Feed Forward and Recurrent NeuralNetworks and Artificial Organisms, Rome, Semeion Software n.12, ver5.0), TasmSABp, TasmSASn (M. Buscema and Semeion Group, 1999, ArtificialNeural Networks and Complex Social Systems [in Italian], Volume 1, Rome,Franco Angeli, pp. 440-464), FF-Bm, FF-Bp, FF-Cm, FF-Sn et al. (D. E.Rumelhart, G. E. Hinton, and R. J. Williams, 1986, Learningrepresentations by back-propagating errors, Nature, 23: 533-536; M.Buscema, 2000, Squashing Theory and Contractive Map Network, Rome,Semeion Technical Paper n. 23i-23e; M. Buscema, 1995, Self-ReflexiveNetworks. Theory, Topology, Applications, in Quality & Quantity, KluwerAcademic Publishers, Dordtrecht, The Netherlands, vol. 29(4), 339-403,November). The publications describing the above neural networks shallbe considered as a part of this invention.

The teaching step consists in generating a database of pixelidentification vectors which are uniquely associated to the type ofobject or quality reproduced by pixels of digital or digitized imageswhich are encoded as described above, and are interpreted on the basisof visual operations performed by qualified personnel. Theidentification vector of each pixel is associated to the type of objector quality of what is reproduced by the pixel, a list of object types orqualities of interest having been previously defined, consistent withthe typical subjects of the digital or digitized images used forteaching, hence for generating the knowledge database to be provided tothe processing algorithm.

The knowledge database for teaching the processing algorithm is providedor allowed to be accessed by the processing algorithm, depending on thespecific teaching mode of the selected processing algorithm.

At the end of the step for teaching the processing algorithm, a digitalor digitized image of an image subject is encoded with the abovedescribed method, in a manner compatible with those used in the imagesdesigned to form the knowledge database and a list of object types orqualities is defined, among those included in the knowledge database ofthe processing algorithm. The processing algorithm substantiallycompares the identification vectors of the individual pixels generatedby the encoding process, and assigns the most probable object type orquality of the reproduced object, to each pixel.

The different indications of object types or qualities associated toeach identification vector for image pixels are then displayed byprinting lists and/or by differentially highlighting, e.g. by colors,the pixels of the image to be processed directly on the image.

Depending on the total number of pixels and on the accuracy required bythe list of object types or qualities reproduced in the individualpixels, either subsets of types or qualities or all the types and/orqualities may be selected.

The digitized or digital images may be two-dimensional orthree-dimensional with reference to what has been described for theencoding method, or may consist each of a sequence of images of the sameframe, as acquired at different instants.

In greater detail, FIG. 4 shows the two teaching and processing steps inwhich 10 denotes a set of digital or digitized images, both individuallyand in the form of image sequences. 11 denotes the procedure of encodingeach pixel of said images into the corresponding identification vector.12 denotes the step of uniquely associating the object quality or typeas reproduced by each pixel to the corresponding identification vectorbased on the list of predetermined object types or qualities 13 and 14denotes the reference or teaching database for the image processingalgorithm.

In the processing step, a digital or digitized image or a set of saidimages, such as a sequence of images of the same frame, denoted as 18,is subjected to a step of pixel encoding into identification vectors,denoted as 19, and the identification vectors are provided to theprocessing algorithm 17, which is also supplied with a list of types orqualities specifically sought for and included in the list 13 wherewiththe teaching database 14 was prepared to be accessed by the processingalgorithm 17. The processing algorithm assigns to each identificationvector for the pixels of the image/s 18 an object type or a quality andthe identification vectors are decoded in 20 into the correspondingpixel, the latter being assigned any pixel aspect changes uniquelyrelated to the type associated thereto, for instance a color or thelike. The pixels marked thereby are displayed on the screen, e.g. overthe original image/s and/or a list of the identification vectors for thepixels of the digital image/s to processed is printed, and/or the imagedisplayed on the screen is printed.

As an alternative thereto, the data provided from the algorithm may beused for further processing, based on the recognition of the objectqualities or types reproduced by the individual pixels thanks to theprocessing algorithm.

Any further processing or handling of the data provided by the algorithmmay be performed by the algorithm itself or by other types ofalgorithms, depending on the desired functions or handling purposes.

Hence, for example, the above processing method may be used to simplyrecognize objects, or qualities or conditions or statuses of the objectsreproduced by pixels.

This type of processing is advantageously used in the medical field, asan automatic support to reading and interpretation of diagnostic images,particularly radiographic images, ultrasound images, Nuclear MagneticResonance images, or the like.

As an alternative thereto or in combination therewith, the method of theinvention may be used to recognize shapes or types of objects in imageswith the same subject and substantially the same frames, but being shotor acquired with different methods. In this case, each of the images ofthe same subject and showing substantially the same frame may beprocessed with the processing method of the invention, whereupon thepixels of the different images, having substantially identical positionstherein and being associated to the same object type or object qualityare shown in overlaid positions, thereby providing an image whichcontains the details of the same subject, as imaged with the threemethods. This may be advantageous to integrate into a single image,details that may only be recognized and reproduced with some of theacquisition or imaging techniques or modes, as well as details that maybe only recognized and imaged with other acquisition or imagingtechniques.

Similarly, the processing method may be used for image correction, e.g.to accurately correct defocused images. Here, by providing an adequateteaching database including the identification vectors of defocusedimages with a uniquely associated object type or quality reproduced bythe corresponding pixel, the inventive method may be used to generate afocused image, by identifying the pixels which reproduce unfocusedborders and removing or modifying them to obtain the focused image.

The two applications as described above may be obviously used in themedical field.

Particularly, by using the processing method to obtain overlaid imagessubstantially of the same frame of the same subject, all data obtainedby different imaging techniques, e.g. ultrasound, radiographic and MRimaging may be integrated into a single image.

It shall be noted that the possibility to recognize the different typesor qualities of an object allows to integrate the information achievedby different imaging techniques even when the latter provides images ofdifferent frames of the same subject, while obviously accounting for therelative arrangement of said different frames in space.

FIGS. 5 to 13 show the results of an embodiment of the inventive methodas applied to the medical field and to the purpose of supporting thediagnostic activity of the physician.

Particularly, the example as shown in FIGS. 5 to 13 relates to the useof the processing method for selective recognition of different types oftissues in diagnostic Nuclear Magnetic Resonance images.

EXAMPLE 1 FIG. 5

SUBJECT BREAST IMAGING METHOD NUCLEAR MAGNETIC RESONANCE PURPOSERECOGNITION OF TISSUE TYPES TISSUE TYPES 1. BENIGN TUMOR 2. MALIGNANTTUMOR PIXEL ENCODING 3 × 3 PIXEL MATRIXES THE ENCODED PIXEL IS THECENTRAL PIXEL IMAGE DEFINITION 256 × 256 PIXEL

In the example 1, a teaching database for the image processing algorithmis generated to recognize two types of tissues, i.e. benign tumor andmalignant tumor in the breast region.

A predetermined number of Nuclear Magnetic Resonance images of thebreast region of patients who have been diagnosed a malignant breasttumor and of patients who have been diagnosed a benign breast tumor arepixel encoded according to the method described above. Theidentification vectors for the pixels have, as components, all thesurrounding pixels in a 3×3 pixel matrix, in which the pixel to beencoded is the central pixel (FIG. 1).

The identification vector for each pixel is assigned the type of tissuereproduced by the pixel in the image.

Therefore, the teaching database for the image processing algorithmcontains identification vectors of image pixels relating to two tissuetypes, i.e. malignant tumor tissues of the breast region and benigntumor tissues of the breast region.

The following algorithms composed of so-called neural networks are usedas processing algorithms.

PROCESSING ALGORITHM MetaGen1 MetaGen MetanetAf FF-Bm MetaBayesMetanetBp MetanetCm FF-Sn TasmSABp TasmSASn FF-BP FF-Cm

A sequence of Nuclear Magnetic Resonance images of the breast region ofdifferent patients, which were not used for generating the teachingdatabase are encoded as disclosed above with reference to FIG. 1, andaccording to the pixel encoding method which is followed to encode theimages used to create the teaching database for the processingalgorithms. An example of these images is shown in FIG. 8. The whitering denotes the presence of benign tumor tissue.

The identification vectors for the individual pixels are provided to theprocessing algorithm for the recognition of the tissue type reproducedthereby.

The algorithm assigns to the different identification vectors, hence tothe corresponding pixels, the type of tissue represented thereby basedon the teaching database.

The result thereof is displayed by appropriately and differentiallycoloring the pixels whereto the type of benign or malignant tumor tissuehas been assigned.

FIG. 9 shows a tissue type recognition result example referred to theimage of FIG. 8, in which the white outlined area had been recognized byvisual analysis as representing the benign tumor tissue.

In FIG. 9, the black screened white zone represents the pixels wheretothe processing algorithms assigned the type of benign tumor tissue.

The white encircled black zones denote the pixels whereto the processingalgorithms assigned the type of malignant tumor tissue.

As compared with the recognition of malignant or benign tumor tissuetypes, obtained through visual analysis by a physician, for some pixelsthe algorithm provided non accordant indications, which have beentemporarily classed as wrong.

FIG. 5 shows both in a data table and in a chart the predictionreliability results of tissue type recognition obtained by processingwith the different neural networks as listed above. The results obtainedtherefrom are expressed in terms of correct benign or malignant tumortissue recognition percentage, of recognition sensitivity and ofweighted and arithmetic correct recognition average, as well as absoluteerrors. The chart only shows the two tissue type recognition percentagesand the errors.

The above description clearly shows the high tissue recognitionreliability obtained by the processing algorithm, hence the highreliability provided by the processing method, when used as a diagnosticimage analysis method, for recognizing and indicating the presence oftypes of tissues.

It is apparent that this method is not a pure diagnostic method,because, although the indications provided thereby are highly reliable,they can provide no diagnostic certainty nor substitute or prevent theexecution of additional specific analyses or examinations needed for atotally reliable diagnosis.

The method substantially provides support to diagnostic image readingand interpretation, aimed at better location and recognition of specifictissue types represented in the images. The difficulties in reading andinterpreting diagnostic images, whether of the MRI and ultrasound orradiographic imaging types are self-evident from FIG. 8.

Obviously, a better result may be achieved by changing the encodingrule, i.e. by increasing the number of surrounding pixels around thepixel to be encoded, which are designed to form the components of theidentification vector of said pixel to be encoded.

Better results were also achieved by increasing the number of the tissuetypes contained in the teaching database, as shown in the followingexamples.

EXAMPLE 2

The example 2 is similar to the example 1, an additional tissue type,i.e. normal tissue, being included in the recognition database. Hence,when the teaching database is generated, the encoded vectors for thepixels of the images are uniquely associated to one of the tissue typesrepresented thereby, i.e. benign tumor tissue, malignant tumor tissue ornormal tissue.

This additional type allows to count on a greater number of pixels andcorresponding identification vectors having a sure meaning. In theprevious example, these pixel identifying vectors and the correspondingpixels have no meaning for the processing algorithm, whereas in thissecond example, the processing algorithm can assign an additionalwell-defined class or type of tissue.

This additional possibility provides an error reduction, since thealgorithm may choose from three conditions or statuses to be assigned tothe identification vectors.

The execution of the example is exactly as described and shown withreference to the previous example 1.

In this example, only the following neural networks were selected asprocessing algorithms:

PROCESSING ALGORITHM MetaGen1 MetaGen MetanetAf MetaBayes MetanetBpMetanetCm FF-Sn FF-BP FF-CmThe numerical results are listed in FIG. 6.

The recognition of tissue types by the different neural networks is morereliable.

EXAMPLE 3

The example 3 is similar to the above examples, but includes five tissuetypes, i.e.: benign tumor tissue, malignant tumor tissue, normal tissue,muscular tissue and image background.

The teaching database is generated as described above with reference tothe previous examples and includes pixel identifying vectors, each beinguniquely assigned one of the above five types, i.e. the one representedby the respective pixel.

The tissue type recognition result per image pixel is shown in FIG. 7and for the following algorithms:

PROCESSING ALGORITHM MetanetAf MetaBayes MetanetBp MetanetCm FF-Bm FF-SnFF-Bp FF-Cm LDA

All these processing algorithms, except LDA, are neural networks. LDA isa discriminating algorithm.

The result achieved thereby proves a very high tissue type recognitionreliability. It shall be further noted that the discriminating algorithmalso provides unexpected results in relation to the capabilitiesthereof, when compared with normal conditions, although the resultsprovided thereby are definitely lower than those obtained by neuralnetworks.

The chart shows the errors for each different algorithm.

FIG. 10 shows an example of result visualization by differentiated pixelcoloring, depending on the different types recognized therefor, and withreference to the example of FIG. 8.

The muscular tissue, the background, the normal tissue and the benigntumor tissue are properly recognized. The indications detected in FIG.9, regarding the example 1, in which in some small portions benign tumortissue was wrongly located, have disappeared herein.

FIGS. 11, 12 and 13 show an example of Nuclear Magnetic Resonancedigital photographs of a breast region including a malignant tumortissue, as highlighted by a white ring in FIG. 11 and by a correspondingpartial enlarged view in FIG. 11.

FIG. 13 shows the result obtained in terms of recognition of the tissuetypes reproduced by the image pixels, thanks to the method of theinvention, with the help of a neural network as a processing algorithm.The teaching database is the same as in the example 3, including allfive tissue types.

Correct recognition of malignant tumor, background, and muscular tissueis apparent from FIG. 13, which shows an example of recognitionprocessing result visualization.

As regards the described examples and the processing method of theinvention, it shall be noted that, while identical frames are alwaysused in the figures of the examples, the described method is notnecessarily limited to the type of frame. Thanks to the fact that theencoding method according to the invention allows to account for therelation between the encoded pixel and the surrounding pixels, theteaching database actually allows to identify and recognize the tissuetype reproduced by a pixel of a digital image of the same anatomicregion or possibly of different anatomic regions, regardless of thespecific image frame.

This considerably simplifies imaging, as it does not require images tobe always acquired with the same frame.

It shall be further noted that the teaching database may dynamicallygrow by the addition of data gathered and confirmed through successiveprocessing procedures. In fact, once the tissue type assignment by thealgorithm to a predetermined pixel and to the correspondingidentification vector has been confirmed, the identificationvector-tissue type pairs so formed may be themselves loaded in theteaching database which grows with the use of the method, thereby makingthe processing algorithm increasingly expert and reducing the indecisionor error margin.

It shall be additionally noted that, by simply adding a tissue type tobe recognized and assigning it to pixel identification vectors of imagesused to generate the teaching database, and which reproduce said tissuetype, the recognition processing may be changed as regards the number ofdifferent tissue types to be recognized.

By totally changing the teaching database, it is possible to recognizeother tissue types in other types of subjects, and to correct or overlayimages of the same subject, obtained with different techniques.

Image processing aimed at recognizing tissue types or qualities is alsopossible by a pixel vector encoding method, which accounts for timevariations of the pixel reproducing a specific object, i.e. the typeaccording to the encoding example of FIG. 3. As disclosed above, thisencoding type allows to encode pixels of image sequences.

In the diagnostic field, a tissue recognition method may be provided formoving subjects, such as in the case of ultrasound imaging or similar,of the heart.

Currently, upon the control of the electrocardiogram, sequences ofultrasound images of the heart region are acquired, to be further storedand displayed successively like in a movie sequence. The interpretationof the images within the sequence, especially with reference to specificdiseases is not easy and is relatively uncertain. Hence, thanks to theprocessing method as described with reference to the examples 1, 2 and3, as combined with a method of encoding image pixels of image sequencesof the same type as described with reference to FIG. 3, an automatedanalysis and recognition of the tissue types reproduced by theindividual pixels of the image sequence may be performed, whereupon saidtypes may be highlighted in a differentiated manner, over the sequenceimages, to improve the viewing and locating potential of the physicianor the qualified personnel.

A similar application field for a combination of the processing methodfor tissue type recognition in digital or digitized image sequences witha pixel encoding method for the pixels of said sequence images,respectively by an identification vector including, for each pixel, thevalues of the pixel to be encoded and of the pixels around it, of eachimage of the image sequence, consists in the recognition of tissues orvascular or lymphatic flows in combination or not with the injection ofcontrast agents, as well as in the recognition and measurement ofcontrast agent perfusion.

In this case, the sequence of images acquired with time after theinjection of contrast agents is encoded with the method as describedwith reference to FIG. 3. The teaching database for the processingalgorithm includes behavior types, e.g. arterial blood flow or lymphaticor venous flow and stationary tissues and/or tissues of vessel walls.Then, the recognition results are displayed e.g. by appropriatelycoloring the pixels relating to the different types.

In all the above cases, it is important to generate an adequate teachingdatabase containing the relevant object types or qualities for thespecific examination.

It shall be noted that the same processing unit, i.e. the hardwarewherein the processing software is loaded, may perform any of the aboverecognition processing procedures, by simply providing the processingsoftware with the proper teaching database for the images to beprocessed and by obviously encoding the images to be processed.

Regarding image correction procedures, i.e. image processing aimed atsuppressing or recognizing artifacts and/or at recognizing defocusedareas and at correcting them by focusing the image, the processingmethod of the invention does not substantially change.

As regards artifacts, a teaching database shall be generated in whichknown images with or without artifacts are encoded, by assigning theartifact type to artifact-reproducing pixels and the correct pixel typeto correct object reproducing pixels. Once an image or a sequence ofimages has been recognized, it may be easily corrected by suppressingartifact-related pixels or by assigning to artifact-related pixels thetissue types or qualities which they might have with reference, forexample, to surrounding pixels.

Defocusing may be corrected in a similar manner.

The processing method of the invention may be also advantageously usedto generate images composed of individual images of the same subject asobtained by different techniques, e.g. Nuclear Magnetic Resonanceimaging, ultrasound imaging and x-ray imaging.

In this case, the teaching database will contain pixel encoding vectorsfor all three images obtained with the three different techniques, withtissue types or qualities corresponding to said pixels being uniquelyassociated to said vectors. Hence, image portions are uniquelyassociated to specific tissue types and said well-defined portions maybe displayed in overlaid positions or other combined arrangements withina single image.

An additional application of the inventive recognition method, incombination with imaging methods, particularly for diagnostic purposes,such as ultrasound or Nuclear Magnetic Resonance imaging methods,consists in that imaging is performed with less accurate butconsiderably faster imaging sequences or techniques and that thedisplayed image is an image processed with the recognition method ofthis invention.

This allows to maintain high reliability levels in the recognition ofthe types or qualities reproduced by the individual image pixels and toshorten the diagnostic imaging times. This arrangement may be veryuseful particularly for ultrasound or Nuclear Magnetic Resonanceimaging, which require relatively long imaging times, in certainsituations, and provides apparent advantages.

The processing method of the invention also provides considerableadvantages in the recognition of tissue types, like potentially diseasedtissues, for example tumor tissues at very early stages. Currently,x-ray mammography, for example, is performed with spatial resolutions ofabout 7 micron. Therefore, these images or the data associated theretohave such a resolution that different tissue types may be discriminatedat very early growth levels, in groups of a few cells. Nevertheless, thehuman eye only has a spatial resolution of 100 micron. Hence theconsiderable imaging resolution cannot be currently used.

Conversely, the method of this invention does not have any spatialresolution limits, except those possibly associated to image digitizingmeans.

Therefore, the spatial resolution limits of the human eye may be loweredby using appropriate image digitizing or digital sampling means, topotentially reach the spatial resolution available at the imaging stage.

Therefore, by using the method of this invention, it is possible togenerate a digitized virtual image which consists of a two-dimensional,three-dimensional or multi-dimensional set, in which the virtual imageis composed of image data for image unit dots relating to the spatialresolution below the one of the human eye.

The processing method essentially includes the same steps as describedabove, i.e. generating pixel encoding vectors and quality and typerecognition processing, particularly for tissues, as described above. Asdescribed herein, the different object types or object qualities may behighlighted by appropriately changing the aspect of the pixel relatedthereto, e.g. by a suitable differentiated coloring arrangement.

From the set of pixel identification vectors, the pixel data matrix maybe reconstructed, and said data may be used to control, for instance, aprinter and/or a display screen.

The printer or display may be controlled in such a manner as to allowthe individual pixels to be also displayed at the resolution of thehuman eye, e.g. by using an image variation in which the data of eachpixel of the high resolution image, i.e. having a resolution below thatof the human eye is used to control a unit group of pixels of thedisplay or printer, whose pixels take the same aspect as thecorresponding pixel to be displayed. The image is inflated rather thanenlarged, each high resolution pixel being represented by displaying apixel sub-matrix which comprises a sufficient number of pixels togenerate an image portion having a resolution of the same order ofmagnitude as the one of the human eye or higher.

In the case of an image composed of pixels whose size corresponds to aresolution of 7 micron, it is possible to form unit groups of 14×14=196pixels, thus simulating a resolution of 98 micron.

Here, the 196 pixels of the unit group are controlled to assume the sameaspect as assigned to the corresponding high definition pixel, therebygenerating an image point which is visible to the human eye.

Obviously, the above displaying steps allow to generate unit groups ofhigh definition pixels which may also have a greater or smaller numberof pixels, substantially corresponding to a greater or smallerenlargement of the individual high resolution pixels.

The enlargement factor may be also user-preset by possibly allowing todelimit or define an image portion whereto the enlargement displayingstep is to be applied and to modify said image portion to be enlargedfor successive and different enlargement steps with differentenlargement and resolution factors.

The above description clearly shows that the recognition methodaccording to this invention, based on the pixel encoding techniqueallows to process, evaluate or provide highly reliable indications, evenregarding such information that is unrecognizable for the human eye.

Obviously, as for the method described above, multiple applicationfields may be provided, with particular reference to diagnostic imageprocessing and to healthy or normal tissue or diseased tissuerecognition, especially for benign and malignant tumor tissues.

In the latter application field, the improvement as described aboveallows to analyze the tissue type and to obtain indications regardingthe presence of benign or malignant tumor tissues at very early stageswhich, at a resolution of 7 micron, are composed of a very small numberof cells.

1. A method for encoding pixels of digitized images comprising:providing one or more images comprising a set of image dots, namedpixels in two-dimensional images and voxels in three-dimensional images,each of said pixels or voxels being represented by a set of values whichcorrespond to a visual aspect of the pixel on a display screen or in aprinted image, characterized in that the pixels (5) or voxels (14) of atleast one portion of interest of the digital or digitized image or eachpixel (5) or voxel (14) of the set of pixels or voxels which form theimage is uniquely identified with a vector whose components are given bythe data of the pixels (5) or voxels to be encoded (14) and by the dataof at least one or at least some or of all of the pixels (1, 2, 3, 4, 6,7, 8, 9; 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19,20, 21, 22, 23, 24, 25, 26, 27) around the pixels to be encoded andarranged within a predetermined subset of pixels or voxels included inthe whole set of pixels or voxels which form the image.
 2. An encodingmethod as claimed in claim 1, characterized in that the components ofthe pixel (5) or voxel (14) identifying vector are determined byselecting, as pixels or voxels surrounding the pixel to be identified,all the pixels or voxels (1, 2, 3, 4, 6, 7, 8, 9; 1, 2, 3, 4, 5, 6 7, 8,9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27)that are directly adjacent to said pixel or voxel to be encoded.
 3. Amethod as claimed in claim 1, characterized in that the components ofthe identification vector of a pixel (5) or voxel to be encoded (14)also consist of at least one or at least some or at least all of thepixels or voxels surrounding the pixels or voxels (1, 2, 3, 4, 6, 7, 8,9; 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20,21, 22, 23, 24, 25, 26, 27) that are directly adjacent to said pixel (5)or voxel to be encoded (14).
 4. A method as claimed in claim 1,characterized in that the components of the identification vector,corresponding to the pixel (5) or voxel to be encoded (14) and to thesurrounding pixels or voxels (1, 2, 3, 4, 6, 7, 8, 9; 1, 2, 3, 4, 5, 6,7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,27) are arranged in such a manner as to correspond to the distancerelation of said pixels or voxels (1, 2, 3, 4, 6, 7, 8, 9; 1, 2, 3, 4,5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,25, 26, 27) with one another and with the pixel (5) or voxel to beencoded (14), with reference to a predetermined reading sequence ofsurrounding pixels or voxels (1, 2, 3, 4, 6, 7, 8, 9; 1, 2, 3, 4, 5, 6,7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,27), selected for forming said identification vector and the pixel (5)of voxel to be encoded (14).
 5. A method as claimed in claim 1,characterized in that the components of the identification vector arearranged in such a manner that the pixel (5) or voxel to be encoded (14)has a central position which corresponds to the one taken in the imagepixel or voxel set, related to the pixels or voxels (1, 2, 3, 4, 6, 7,8, 9; 1, 2, 3, 4, 5, 6, the pixel (5) or voxel to be encoded (14) whichhave been selected for determining the identification vector components.6. A method as claimed in claim 1, characterized in that it includes thestep of encoding a sequence of digital or digitized images of a singlesubject and relating to a single frame of said subject, which sequenceincludes at least two images acquired with a time interval therebetween,which identification vector for a pixel (5) or voxel (14) to be encoded,having the same position in the pixel matrix which forms said sequenceimages, is formed by the value of said pixel (5) or voxel to be encoded(14) and by the surrounding pixels or voxels selected to form thecomponents of said identification vector for each image that is part ofsaid image sequence.
 7. A method as claimed in claim 6, characterized inthat the identification vector for a pixel (5) or voxel to be encoded(14) within a sequence of digital or digitized images includes thevalues of said pixel or voxel to be encoded and of the pixels or voxelsselected to form the components of said identification vector of all theimages of said sequence, the values of the pixel (5) or voxel (14) to beencoded and of the pixels or voxels around it, selected to form thecomponents of the identification vectors, being ordered with respect tothe instant whereat the individual images of the sequence were acquired,in such a manner as to form subsets of identification vector components,referred to the same image of the image sequence or to the sameacquisition instant.
 8. A method as claimed in claim 7, wherein thesubsets of identification vector components referred to the differentimages of an image sequence are successively ordered with reference tothe instant whereat the corresponding image of the image sequence to beencoded was acquired.
 9. A method of processing digital or digitizedimages, operating based on image pixel or voxel encoding as claimed inclaim 1, and characterized in that it includes generation of a teachingdatabase and teaching of the processing system, including the followingsteps: encoding a predetermined number of digital or digitized imagesinto pixel or voxel identifying vectors; uniquely associating eachidentification vector to the corresponding type of object or to thecorresponding quality, as determined by traditional image analysis andactually reproduced by each pixel or voxel encoded by the correspondingidentification vector, with reference to a list of predetermineddifferent types or qualities generating a teaching database for aprocessing system, which database comprises the binomials formed by saididentification vectors and by the associated type or quality of theobject reproduced by the corresponding pixel or voxel; actually teachingthe processing system, by entering and loading the teaching databasetherein or by allowing the processing system to access the database; arepeatable processing step for different images or image sequences withno need to repeat the teaching step, and comprising the following steps:encoding the pixels or voxels of a not otherwise evaluated image byidentification vectors associated to each pixel or voxel; entering saidimage pixel or voxel identifying vectors in the processing system toobtain, at the output of said processing system and as a result of theprocessing, the type or quality of the object represented by each imagepixel or voxel, with reference to the object types or qualities includedin the teaching database.
 10. An image processing method as claimed inclaim 9, characterized in that the processing system consists of analgorithm for comparing the pixel identifying vectors of the teachingdatabase with the pixel identifying vectors of the encoded images to beprocessed or of the sequence of encoded images to be processed.
 11. Amethod as claimed in claim 9, characterized in that the processingsystem consists of a discriminating algorithm, of the type known as LDA.12. A method as claimed in claim 9, characterized in that the processingsystem consists of an algorithm known as a neural network.
 13. A methodas claimed in claim 9, characterized in that the pixels or voxels of theprocessed image wherefor an object type or quality has been recognizedare displayed differentially from each other and from the image for eachobject type or quality option.
 14. A method as claimed in claim 13,characterized in that the pixels or voxels of the processed imagewherefor an object type or quality has been recognized are displayeddifferentially from each other and from the image for each object typeor quality option, and over the original image.
 15. A method as claimedin claim 14, characterized in that the original image is displayed in amonochromatic mode, particular in black and white or a gray scale.
 16. Amethod as claimed in claim 9, characterized in that the results of imageprocessing are stored in the teaching database for the processingsystem.
 17. A method as claimed in claim 16, characterized in that,before being stored in the teaching database, the image processingresults are validated by a visual control and/or other analysis means.18. A method as claimed in claim 16, characterized in that imageprocessing results are stored in the form of identification vectors forthe pixels or voxels of the processed image, associated to the objecttype or quality assigned thereto upon processing.
 19. A method asclaimed in claim 9, characterized in that it is a method for recognizingtypes of objects reproduced by image pixels or voxels.
 20. A method asclaimed in claim 9, characterized in that it is a method for measuringcontrast agent perfusion, wherein a sequence of ultrasound or NuclearMagnetic Resonance images of a predetermined anatomic part of a patientare detected after injecting so called contrast agents in said anatomicpart, which method includes the following steps: generating a teachingdatabase for the expert processing system comprising identificationvectors for pixels or voxels or image sequences obtained when contrastagents are present, whereto a quality or type o perfusion behavior isassociated, among different typical perfusion types or qualities;actually teaching the processing system, by entering or handling data ofthe teaching database; acquiring a sequence of images of an anatomicpart after injecting contrast agents therein, and encoding the pixels orvoxels of the images of said sequence, into identification vectors forthe pixels of said image sequence; processing by the identificationvector processing algorithm, which associates, based in the teachingdatabase, a perfusion behavior type or a perfusion quality to eachidentification vector, hence to each pixel or voxel of the imagesequence; displaying the image sequence, and highlighting the pixels orvoxels associated to the different perfusion behavior qualities or typesby means for unique visual aspect characterization of said pixels orvoxels.
 21. An image processing method as claimed in claim 9, whichincludes an image pixel or voxel encoding method, characterized in thatit is a method for recognizing and displaying parts of moving organs orphysiological structures, particularly of the heart, wherein a sequenceof ultrasound or radiographic or Nuclear Magnetic Resonance images ofthe heart or of any other organ or physiological structure is acquired,which method includes the following steps: generating a teachingdatabase in which each identification vector for pixels or voxels of a.plurality of image sequences of the heart or any other organ orphysiological structure is assigned the type or quality of what isreproduced by the corresponding pixel or voxel; actually teaching theprocessing system, by entering or handling data of the teachingdatabase; encoding a sequence of images of the heart or any other organor physiological structure for further processing; processing saidencoded sequence of images so that the processing algorithm may assign,based on the teaching database, the type or quality reproduced by eachpixel or voxel of the encoded image sequence; displaying the result andvisually highlighting the pixels of voxels corresponding to specifictypes or qualities by uniquely changing the aspect of these pixels ofvoxels according to each of the specific types or qualities.
 22. Amethod as claimed in claim 9, wherein the following steps are included:generating a teaching database by encoding image pixels or voxels intoidentification vectors, and wherein each identification vector for thepixels or voxels of said images is assigned the type or quality whichdefines the presence or absence of the image defect or aberrationdepending on whether the corresponding pixel or voxel reproduces or hasor not said aberration or said defect; actually teaching the processingsystem, by entering or handling data of the teaching database; encodingimages; processing said encoded images so that the processing algorithmmay assign, based on the teaching database, the type or quality whichdefines the presence or absence of an image defect or aberration foreach pixel or voxel of the encoded images; displaying the result andvisually highlighting, by aspect change arrangements, the pixel/s orvoxel/s which have been assigned the type which defines the presence ofaberrations or defects and possibly indicating the quality of theaberration or defect assigned to a pixel or voxel, as distinct from theone assigned to other pixels or voxels, by further aspectdifferentiation of the pixel/s or voxel/s, uniquely related to thedifferent defect or aberration qualities.
 23. A method as claimed inclaim 22, characterized in that it further includes defect removal,according to the following steps: adding to the teaching database pairsof encoded images, which have or do not have image defects oraberrations, by associating the identification vectors with thecorresponding types defining the presence or absence of pixelaberration; encoding the pixels or voxels of an image and processing thelatter to assign the type that defines the absence of presence ofaberrations or defects, and possibly the quality of said aberrations ordefects to each pixel or voxel of the image; correcting the aspect ofthe pixels or voxels which have been found to have defects oraberrations by assigning them the aspect of the defect- oraberration-free pixels or voxels of the image, which is coupled, in theteaching database, to the corresponding image which has said aberrationsor defects.
 24. A method as claimed in claim 22, characterized in thatthe processed images are previously or subsequently processed forspecific recognition of the object types represented by the pixels. 25.A method as claimed in claim 22, characterized in that the type ofdefect or aberration is a defocusing defect and/or an artifact and/or awrong exposure and/or a defective development.
 26. A method as claimedin claim 9, characterized in that it is a method of overlaying digitalor digitized images of the same subject, obtained by different imagingtechniques, which includes the following steps: encoding each of theimages of the same subject, obtained with different imaging techniques,processing each of the images of the same subject, obtained withdifferent imaging techniques, for recognizing types of objects orqualities; combining the information provided by the pixels of thedifferent images, which are assigned to the same type of object, into asingle image.
 27. A method as claimed in claim 1, characterized in thatit is a method for processing digital or digitized diagnostic images,aimed at the recognition of at least one type of tissue or anatomic orphysiologic object or one quality thereof.
 28. A method as claimed inclaim 27, characterized in that it is a diagnostic image processingmethod for recognizing and discriminating benign tumor tissues andmalignant tumor tissues, as reproduced by the pixels or voxels of thediagnostic images, the teaching database being composed ofidentification vectors for image pixels or voxels that represent saidmalignant and benign tumor tissues, uniquely associated to thecorresponding tissue type.
 29. A method as claimed in claim 28,characterized in that it is a method for recognizing and discriminatingbenign tumor tissues, malignant tumor tissues and normal tissues, asreproduced by the pixels or voxels of the diagnostic images, theteaching database being composed of identification vectors for imagepixels or voxels that represent said malignant and benign tumor andnormal tissues, which vectors are uniquely associated to thecorresponding tissue type reproduced by the pixels or voxels encodedinto said vectors.
 30. A method as claimed in claim 29, characterized inthat it is a method for recognizing and discriminating benign tumortissues, malignant tumor tissues, normal tissues and muscular tissues asreproduced by the pixels or voxels of the diagnostic images to beprocessed, the teaching database being composed of identificationvectors for image pixels or voxels that represent malignant and benigntumor tissues, normal and muscular tissues, which vectors are uniquelyassociated to the corresponding tissue type reproduced by the pixels orvoxels encoded into said vectors.
 31. A method as claimed in claim 28,characterized in that it is a method for recognizing and discriminatingbenign tumor tissues and/or malignant tumor tissues and/or normaltissues and/or muscular tissues and image background, as reproduced bythe pixels or voxels of the diagnostic images to be processed, theteaching database being composed of identification vectors for imagepixels or voxels that represent malignant and/or benign tumor tissues,and/or normal and/or muscular tissues and image background, whichvectors are uniquely associated to the corresponding tissue type orbackground reproduced by the pixels or voxels encoded into said vectors.32. A method as claimed in claim 1, characterized in that the digitizedimage is an image whose pixel size corresponds to a high resolution,below human eye resolution, the pixel data which are processed forrecognition, being used to control all the pixels of a high resolutionpixel unit group which has such a number of pixels that the aspect ofall the pixels of each pixel unit group is identical to that of the highdefinition pixel associated thereto and the displayed or printed imageof said pixel unit group may be viewed or detected at the human eyeresolution or worse.
 33. A method as claimed in claim 32, characterizedin that the number of high definition pixels which form the pixel unitgroup is adjustable and allows to define different enlargement levels.34. A method according to claim 33, charaterised in that as a value ofthe minimum image area it is used the mean of the brightness values ofthe single pixels forming the minimum image area and/or the variancethereof and/or additionally the mean and or the variance value of thecolour values of the single pixels.
 35. A method according to claim 1characterised in that instead of considering an image formed by singlepixels, for coding purposes a minimum image area of different size maybe defined which is formed by a predefinite certain number of adjacentpixels, the value of the said minimum image area formed by the saidcertain number of adjacent pixels being calculated as a linear or nonlinear combination and/or as the value of one or more statisticalfunctions of the of the brightness and or color values of the singlepixels forming the said minimum image area.
 36. A method according toclaim 35, characterized in that also the time dependency of the valuesrelating to brightness and/or color of the single pixels forming theminimum image area is added, the time dependency of the combinationsand/or of the statistical functions of the brightness and or colorvalues of the single pixels being used.
 37. A method according to claim35, characterized in that there are defined minimum neighborhood imagearea zones of the minimum image area to be coded each of which minimumneighborhood image area zones may be formed by a single selected pixelsin the neighborhood of the minimum image area to be coded or as acertain number of said pixels in the neighborhood of the said minimumimage area to be coded.
 38. A method according to claim 37,characterized in that the minimum neighborhood image area ones of theminimum image area to be coded have a number of pixels for each zonewhich is identical or different form the number of pixels forming theminimum image area to be coded.